A Coupled Helmholtz Machine for PCA

In this letter we present a coupled Helmholtz machine for principal component analysis (PCA), where sub-machines are related through sharing some latent variables and associated weights. Then, we present a wake-sleep PCA algorithm for training the coupled Helmholtz machine, showing that the algorithm iteratively determines principal eigenvectors of a data covariance matrix without any rotational ambiguity, in contrast to some existing methods that performs factor analysis or principal subspace analysis. The coupled Helmholtz machine provides a unified view of principal component analysis, including various existing algorithms as its special cases. The validity of the wake-sleep PCA algorithm is confirmed by numerical experiments. Indexing terms: Dimensionality reduction, Helmholtz machines, PCA. to appear in Electronics Letters Please address correspondence to Prof. Seungjin Choi, Department of Computer Science, POSTECH, San 31 Hyoja-dong, Nam-gu, Pohang 790-784, Korea, Tel: +82-54-279-2259, Fax: +82-54-279-2299, Email: [email protected]